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Abstract 

Objectives:  The  objective  of  this  project  was  to  develop  and  implement  a  seven 
module  ambulatory  care  continuing  medical  education  (CME)  curriculum  and  to 
evaluate  the  effectiveness  of  this  curriculum  in  facilitating  patient  safety. 
Methods:  For  knowledge  transfer  and  adoption,  each  module  focused  on  one 
safety  concept,  followed  by  clinical  cases  and  take-home  points.  The 
dissemination  strategy  relied  on  the  “train  the  trainer”  model  and  utilized  the 
governance  structure  of  the  American  College  of  Physicians  (ACP).  Results:  By 
April  2004,  some  7,300  internists  had  been  exposed  to  the  curriculum  via  regional 
ACP  Chapters.  We  anticipate  100  percent  chapter  exposure  by  2005.  Findings 
from  the  pre  and  post  CME  analyses  showed  improvement  in  attitudes  towards 
patient  safety;  however,  6-month  follow-up  showed  mixed  results.  To  round  out 
the  picture,  we  attempted  to  measure  the  effectiveness  of  the  curriculum  via  the 
presenters’  evaluation  of  the  curriculum.  Conclusions:  It  was  effective  to  have  a 
trusted  colleague  conveying  the  patient  safety  messages  via  clinical  scenarios 
accompanied  by  take-home  points.  An  added  bonus  was  the  unanticipated  wider 
dissemination  of  the  curriculum  beyond  the  original  target  population. 


Introduction 

The  patient  safety  ambulatory  care  curriculum  is  the  product  of  a  3 -year  grant 
funded  by  the  Agency  for  Healthcare  Research  and  Quality  (AHRQ). 
Programmatically,  it  falls  under  AHRQ’s  overall  patient  safety  initiative  aimed  at 
medical  societies  and  curriculum  development.  The  ambulatory  care  curriculum 
encompasses  the  seven  areas  identified  as  essential  for  improving  safety  by  the 
Institute  of  Medicine  in  its  seminal  report,  To  Err  Is  Human}  As  the  title  of  the 
training  suggests,  Patient  Safety:  The  Other  Side  of  the  Quality  Equation,  the 
curriculum  firmly  identified  patient  safety  as  an  integral  component  of  quality. 
This  was  done  to  link  the  new  conceptual  ideas  and  behavior  patterns  (patient 
safety)  to  accepted  and  proven  methods  (quality  initiatives)  in  the  field  of 
medicine.  As  Rogers"  argues,  individuals  have  to  be  predisposed  to  accept  new 
knowledge.  By  linking  the  new  knowledge  to  accepted  patterns  of  professional 
behavior,  predisposition  was  increased. 

The  curriculum  includes  modules  on  the  following  topics: 

•  Systems 

•  Medication  errors 
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•  Electronics 

•  Communication 

•  The  patient  role 

•  Office  design 

•  Cognition 

Briefly,  the  systems  module  exemplifies  the  powerful  influence  of  systems  on 
patient  safety.  The  medication  errors  module,  in  turn,  presents  proven  strategies  to 
prevent  the  most  common  medication  errors  by,  for  example,  substituting 
computerized  physician  order  entry  systems  for  handwriting.  The  electronics 
module  focuses  on  the  supportive  role  electronics  can  play  as  point-of-service 
decision  tools,  etc.  The  communication  module  identifies  communication  barriers 
and  ambiguities  between  physicians  and  other  health  care  providers  as  well  as 
with  patients.  The  module  also  contains  strategies  for  overcoming  these 
communication  shortcomings.  The  patient  role  module  points  to  patients  as  allies 
in  patient  safety  and  presents  what  they  can  do  to  improve  outcomes.  The  office 
design  module  features  micro  systems  process  design  principles  that  facilitate  safe 
and  reliable  care.  Finally,  the  cognition  module  details  people’s  innate  capacity  to 
process  information  and  cope  under  conditions  of  information  overload  and  stress. 

No  one  module  gives  an  integrated  overview  of  all  seven  areas,  though  each 
module  briefly  identifies  the  other  content  areas  and  notes  where  they  can  be 
found  in  the  curriculum.  Module  structure  is  identical  for  all  seven  content  areas. 
Each  module  has  three  distinct  components:  theory,  clinical  cases,  and  take-home 
points.  For  example,  the  systems  module  opens  with  an  overview  of  systems 
theory  and  its  implications  for  patient  safety.  The  same  is  the  case  for  the 
cognition  and  communication  module.  The  second  component  of  the  modules 
focuses  on  several  clinical  cases  that  present  concrete  examples  of  how  the 
theoretical  systems’  underpinnings  play  themselves  out  in  reality.  Each  case 
illustrates  examples  of  how  safety  was  threatened  from  a  systems  perspective.  In 
most  instances,  cases  exemplify  several  contributing  causative  features.  This 
highlights  the  fact  that  many  patient  safety  problems  can  be  avoided  if  only  one  of 
the  contributing  causes  is  removed.  The  cases  are  selected  so  physicians  can 
identify  with  them  and  recall  similar  experiences.  When  this  happens,  the 
theoretical  concept  is  transformed  into  concrete  reality  and  becomes  actionable  or 
enters  Bloom’s  application  level.  An  analysis  of  what  went  wrong  follows  the 
case,  which,  according  to  Bloom,  is  the  analysis  level.  By  recognizing  what  went 
wrong,  physicians  can  avoid  similar  errors  in  the  future.  The  final  component  of 
each  module  is  the  take-home  point  section,  which  is  designed  to  enable 
physicians  to  implement  safety  changes  in  their  offices  almost  immediately.  The 
take-home  points  constitute  the  Bloomian  synthesis  level,  where  solutions  to 
problems  are  presented.  It  is  important  to  note  here  that  the  solutions  are  not 
resource  intensive  but  represent  easy  to  implement  steps  that  are  within  the 
physicians’  immediate  power. 
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Although  each  module  can  stand  on  its  own,  three  common  themes  run 
through  all  the  modules.  The  themes  are  systems  thinking,  communication,  and 
cognition.  Clinical  cases,  therefore,  can  be  analyzed  from  a  variety  of  theoretical 
perspectives.  The  layering  of  multiple  potential  causes  reinforces  the  complexity 
of  seemingly  simple  health  care  interventions  or  decisions.  Each  decision  draws 
on  past  clinical  experience,  present  assessment,  and  future  expectation  of 
outcome.  When  appropriate,  each  case  features  the  contributory  roles  of  the  many 
individuals,  such  as  technicians  and  nurses  aides,  who  are  involved  in  patient 
safety  issues,  sometimes  behind  the  scenes.  All  of  these  pieces  are  embedded  in 
the  organizational  structure  of  a  clinical  environment  with  rules  and  regulations 
that  can  further  complicate  the  system  thinking. 


Implementation  design 

The  curriculum  was  disseminated  through  the  American  College  of 
Physicians’  (ACP)  chapter  structure.  College  membership  in  the  United  States  is 
organized  into  67  regional  chapters.  We  solicited  the  governors  of  each  chapter  to 
designate  a  leader  for  training  in  patient  safety.  Each  of  these  leaders  was  trained 
in  the  content  of  all  seven  modules  during  a  day-and-a-half  long  workshop  at 
College  headquarters  in  Philadelphia.  We  based  our  training  on  the  “train  the 
trainer”  model  used  in  the  education  for  physicians  on  the  end-of-life  care 
project.4  These  trained  individuals  went  back  to  their  chapters  and  presented  a 
module  of  their  choice  for  continuing  medical  education  (CME)  in  the  scientific 
programs  at  their  chapter  meetings.  To  facilitate  chapter  buy-in  we  provided  the 
patient  safety  presenters,  who  can  be  considered  early  adopters  or  knowledge 
brokers,5  with  the  lecture  materials  in  PowerPoint8’  format  on  a  CD,  plus  speaker 
notes  and  the  take-home  points  as  a  laminated  pamphlet  to  be  distributed  to 
attendees.  Chapter  trainees  received  questionnaires  to  be  filled  out  before  and 
after  the  patient  safety  presentation. 


Method 

Knowledge  dissemination 

American  College  of  Physicians  chapter  members  are  the  target  audience  for 
this  intervention.  For  knowledge  transfer  and  adoption,  each  module  focuses  on 
one  safety  concept,  followed  by  clinical  cases  and  take-home  points,  as  already 
stated.  Concentrating  on  safety  and  its  ties  to  quality  avoids  the  “blame  trap”6  and 
opens  the  door  to  a  more  constructive  dialogue  in  the  discussion  of  the  clinical 
cases.  During  presentations,  physicians  are  encouraged  to  engage  in  case-based 
problem  solving.  This  reinforces  the  theoretical  constructs  and  embeds  them 
firmly  in  clinical  reality.  By  actively  dissecting  the  various  strands  that  contribute 
to  a  safety  problem,  physicians  learn  to  appreciate  the  multifactorial  causation 
common  to  most  safety  breakdowns.  They  also  leam  that  if  just  one  of  the 
causative  factors  had  changed  from  a  negative  to  a  positive  influence,  the  problem 
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could  have  been  avoided  (such  as  when  a  patient  mentions  his/her  allergy  to 
penicillin).  Most  importantly,  however,  by  analyzing  the  cases  from  the 
perspective  of  their  own  experience,  physicians  establish  the  crucial  link  between 
patient  safety  research  and  their  own  reality.  In  effect,  they  rephrase  and 
restructure  patient  safety  concepts  into  their  own  cognitive  schemata.7 

The  take-home  points  encourage  physicians  to  test  and  apply  the  new  patient 
safety  strategy  in  their  clinical  environment.  The  ability  to  test  new  safety 
behavior  via  individual  take-home  points  in  small  increments  makes  it  easier  to 
adopt  and  accommodate  than  if  an  extensive  revision  of  their  office  practices  had 
been  required.  Change  has  been  simplified  to  bite-size  adaptation.8 

The  College  as  change  agent 

The  College  was  the  change  agent,  functioning  at  the  second  level  of  the 
outcomes  research  diagram,  as  seen  by  Streyer  et  al.9  The  demonstration  of  the 
effectiveness  of  the  curriculum  will  be  found  on  the  impact  on  clinical  practice, 
which  is  level  three.  Streyer  et  al.  and  Rogers2  recognize  the  importance  of  time  in 
diffusion  and  adoption.  While  Streyer  et  al.  acknowledge  the  “incremental  and 
circuitous  nature  of  research,”  Rogers  poses  an  S-shaped  curve,  depicting 
cumulative  adopters  over  time. 

The  College  leadership  fully  supported  the  development  and  dissemination  of 
the  curriculum  to  its  members.  The  three  co-principal  investigators  were  the  then- 
Chair  of  the  Board  of  Regents,  Dr.  Rosof;  President  of  the  College,  Dr.  Hall;  and 
Chair  of  the  Governors,  Dr.  Kirk.  Their  position  of  trust  and  status  among 
members  raised  the  visibility,  as  well  as  the  desirability,  of  the  program.  As 
research  in  guideline  dissemination  and  adoption  has  amply  demonstrated,10 
physicians  frequently  rely  on  and  trust  the  judgment  of  their  peers  in  areas  they 
are  unfamiliar  with.  Rogers"  identifies  the  crucial  role  of  exemplary 
demonstrations  as  potent  diffusion  agents. 

The  four  training  sessions  at  College  headquarters  in  Philadelphia  fall  under 
the  exemplary  demonstration  category.  Seventy-eight  individual  chapter 
members,  selected  by  their  Governors,  acted  as  presenters  or  detail  persons.11 
They  spent  one  and  one  half  days  learning  the  entire  seven  module  curriculum. 

All  training  sessions,  save  the  first  one,  featured  the  entire  seven  module 
curriculum.  The  first  session  took  place  when  only  three  modules  had  been 
developed.  The  individuals  who  attended  the  first  training  session  had  a  second 
training  session  to  gain  command  of  the  remaining  four  modules.  Training  took 
place  in  the  College  boardroom,  which  otherwise  is  reserved  for  board-of-regents 
meetings.  Participants  sat  in  an  oval  so  that  everyone  could  see  each  other. 
Individual  module  content  was  delivered  by  the  expert  authors  of  the  modules. 
One  quarter  of  the  time  was  devoted  to  coach  the  presenters  in  how  to  deliver  the 
material  once  they  were  training  their  chapters.  Open  interchange  and  questions 
were  welcomed.  Individuals  left  with  the  assurance  that  we  would  facilitate  the 
entire  presentation,  beginning  with  resource  material,  access  to  further 
background  information,  and  retraining  if  needed.  Over  time  each  training  class 
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developed  a  sense  of  cohesion  and  adopted  a  new  professional  identity  as  patient 
safety  ambassadors. 

Presenters  chose  one  module  for  their  chapter’s  annual  meeting.  Although  the 
original  intent  of  the  grant  was  to  showcase  at  least  three  modules  at  a  time  in  a 
half-day  session,  this  was  not  possible  because  of  time  constraints.  All  chapters 
opted  for  training  in  one  module  at  a  time,  though  some  had  a  later  presentation  of 
another  module. 

Survey  assessment 

Participants  in  the  CME  patient  safety  programs  offered  at  College  chapter 
meetings  were  self-selected.  CME  patient  safety  attendees  received  three  self- 
administered  questionnaires:  1)  a  baseline  questionnaire  immediately  prior  to 
taking  the  course;  2)  an  abbreviated  version  of  the  same  instrument  at  the 
conclusion  of  the  class;  and  3)  a  6-month  follow  up  questionnaire.  Briefly,  the 
instruments  focused  on  attitudes  towards  patient  safety;  the  likelihood  of 
physicians  accepting  the  possibility  of  medical  errors  in  their  own  clinical 
practice;  a  sense  of  control  over  the  occurrence  of  medical  errors;  intent  to  adopt 
various  technologies  to  reduce  the  risk  of  errors;  and,  finally,  the  degree  to  which 
the  study  population  had  been  aware  of  other  sources  of  patient  safety 
information. 


Results 

Knowledge  dissemination  via  the  College  as  change  agent 

We  held  four  training  programs  between  January  2002  and  March  2004.  As 
previously  mentioned,  the  first  training  session  covered  only  three  modules,  while 
the  remaining  three  sessions  featured  all  seven  modules.  To  date  we  have  trained 
78  chapter  representatives.  Some  representatives  are  counted  twice,  because 
presenters  who  attended  the  first  training  session  came  to  a  second  session  to 
learn  the  subsequent  four  modules.  Hence,  there  were  more  participants  (78)  than 
the  number  of  chapters  (67).  As  of  July  2004,  45  chapters  gave  presentations  to 
some  7,500  physicians.  Sixteen  chapters  out  of  these  45  chapters  presented  a 
patient  safety  program  twice.  The  rest  of  the  chapters  had  their  presenters  trained, 
but  have  not  yet  held  their  annual  scientific  program  meeting.  Thus,  we  anticipate 
22  additional  presentations  through  June  2005.  At  that  point,  the  goal  of  reaching 
all  ACP  chapters  in  the  United  States  will  have  been  met.  Over  the  next  several 
years,  the  patient  safety  curriculum  will  remain  a  vital  resource  for  all  1 10,000 
members  of  the  College. 

Presenters’  curriculum  evaluation 

Slightly  more  than  halfway  through  the  grant  cycle  we  wanted  to  learn  what 
presenters  thought  of  the  curriculum  and  why  they  chose  a  particular  module. 
While  recognizing  the  survey  limitations  due  to  the  small  sample  size,  we  still  felt 
we  could  gain  some  understanding.  Therefore,  in  the  fall  of  2003  we  surveyed  the 
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48  presenters  who  had  participated  up  to  that  point  in  order  to  get  their  feedback 
on  the  program.  The  initial  response  rate  was  low,  so  at  a  later  date  we  contacted 
nonresponders  again  while  including  presenters  who  had  been  added  since  the  fall 
of  2003.  Thus,  now  we  have  responses  from  34  individuals,  and  our  response  rate 
is  44  percent  of  a  total  of  78  presenters. 

The  presenters’  survey  contains  12  question  areas.  Most  questions  are  open 
ended  as  we  did  not  want  to  lead  presenters’  answers.  The  questions  range  from 
which  module  they  chose  and  why  to  how  they  feel  about  the  program  in  general 
and  how  they  would  like  to  see  it  improve.  Included  also  were  the  obvious 
questions  concerning  the  strengths  and  weaknesses  of  the  curriculum. 

The  survey  indicates  that  the  medication  error  module  was  the  most  popular. 
This  module  was  followed  in  popularity  by  systems,  idealized  office  design, 
communication,  and  the  role  of  the  patient.  Recently,  several  chapters  offered  to 
present  training  on  the  electronics  and  cognition  modules  in  the  near  future.  In  the 
assessment  of  presenters,  all  modules,  save  the  electronic  module,  have  stood  the 
test  of  time.  The  electronic  field  has  moved  ahead  so  fast  that  the  module  needs 
updating. 

The  College  standard  CME  evaluations — not  to  be  equated  with  the 
specifically  designed  curriculum  questionnaires — corroborated  the  presenters’ 
assessment  of  the  program.  Systems  and  medication  errors  modules  received  the 
highest  approval  rating  from  the  audience.  The  remaining  modules  were  scored  in 
the  low  fours  and  below  on  the  Likert  Scale.  Still,  it  has  to  be  noted  that  these 
findings  are  preliminary,  as  less  than  half  of  all  chapter  presentations  have  been 
tabulated. 

The  most  important  factor  contributing  to  presenters’  choice  of  module  was 
their  comfort  level  with  the  material,  followed  by  how  relevant  they  thought  the 
topic  was  for  their  particular  audience.  The  final  contributing  factor  was  the 
decision  of  the  chapter’s  scientific  program  committee. 

Presenters  felt  that  practicality  served  as  the  core  strength  of  the  modules, 
especially  when  applied  to  medication  errors,  systems,  and  idealized  office 
design.  The  clinical  cases  were  deemed  to  be  particularly  relevant.  The 
medication  errors  module  was  singled  out  for  its  “ease  of  presenting,  its  non- 
controversial  nature,  its  reliability  of  material,”  and  the  low  cost  of  applying  the 
take-home  points.  Two  individuals  described  the  material  as  “eye-opening.” 
Twelve  presenters  stated  that  the  material  provided  a  good  overall  view.  The 
cognition  module  was  judged  to  be  “too  arcane,”  and  “for  geeks  only.”  Presenters 
may  have  felt  intimidated  by  the  link  to  cognitive  science  and  the  need  to  self- 
assess  their  decisionmaking  style  from  the  cognitive  perspective. 

Presenters  felt  strongly  that  they  wanted  to  add  vignettes  from  their  own 
experience  to  get  their  points  across,  something  that  was  not  possible  as  long  as 
we  were  collecting  survey  data.  This  urge  to  add  their  own  stamp  to  the  material 
may  have  been  one  of  the  motivating  factors  to  branch  out  into  nongrant  related 
patient  safety  discussions  and  presentations. 
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Answers  to  the  question,  “How  would  you  improve  this  module?”  were 
varied.  Respondents  felt  that  the  modules  could  be  improved  by  adding  updates  or 
more  recent  requirements,  such  as  those  from  the  Joint  Commission  on 
Accreditation  of  Healthcare  Organizations  (JCAHO).  Respondents  wanted  more 
examples,  especially  for  the  medication  module.  Some  suggested  combining 
related  modules  into  one  by  picking  the  best  from  each.  Nine  respondents  judged 
the  overall  curriculum  as  excellent.  One  respondent  wrote,  “Excellent  program. 
I’ve  learned  a  lot  and  improved  the  safety  of  my  practice.”  Another  comment 
illustrates  the  applicability  of  this  curriculum  to  a  hospital  even  though  the 
material  was  developed  with  ambulatory  care  in  mind.  The  respondent 
commented,  “As  CEO  and  president  of  the  hospital,  patient  safety  is  one  of  my 
top  priorities.  . .  .1  have  newly  established  a  position  of  a  vice  president  (patient 
safety)  in  my  hospital. ...” 

Dissemination  to  other  venues 

Apart  from  regular  chapter  presentations,  24  of  the  34  survey  responders 
reported  presenting  the  material  at  other  venues  such  as  grand  rounds,  residency 
programs,  and  medical  societies  and  other  community  settings.  The  material  also 
was  featured  at  each  ACP  annual  session  from  2002  through  2004.  In  addition,  19 
interested  parties  learned  of  the  program  from  colleagues  and  requested  copies  of 
the  curriculum.  The  requests  came  from  several  quality  assurance  programs  and 
the  Air  Force.  Finally,  the  American  College  of  Cardiology-ACP  joint 
symposium  in  2003  featured  the  medication  error  module.  Recently,  the  author 
presented  the  essence  of  the  curriculum  at  Loma  Linda  University  and  in  the 
planning  session  program  of  the  American  Board  of  Medical  Specialties,  the 
American  Board  of  Family  Practice,  the  American  Board  of  Internal  Medicine, 
and  the  American  Board  of  Pediatrics. 

The  wide  dissemination  and  interest  in  the  modules  beyond  College  chapters 
was  not  anticipated.  The  fact  that  it  occurred  has  to  be  attributed  to  a  general  need 
and  desire  to  become  more  knowledgeable  about  patient  safety.  The  key  factor, 
however,  was  the  commitment  of  the  chapter  trained  presenters.  The  training 
process  resulted  in  a  cadre  of  dedicated  speakers.  These  individuals  are  not  only 
the  early  adopters2,  but  also  knowledge  brokers  and  boundary  spanners,  as  Lomas 
emphasized/  The  patient  safety  message  and  behavior  have  become  a  part  of  their 
professional  identity.  They  reaffirm  this  identity  each  time  they  present  the 
material  anew. 

Future  curriculum  dissemination  possibilities  include  incorporation  of  some  of 
the  material  in  a  joint  American  Board  of  Medical  Specialties  and  American 
Board  of  Internal  Medicine  recertification  program;  and  a  patient  safety  CME  for 
renewal  of  State  licenses.  The  American  Academy  of  Neurology  has  signed  a 
statement  that  they  would  like  to  use  this  ambulatory  care  curriculum,  with 
attribution  to  AHRQ  and  the  College,  if  funding  can  be  obtained  to  adapt  the 
material  to  neurology. 

While  the  dissemination  model  with  the  College  as  change  agent  worked  well, 
especially  as  the  dissemination  was  mediated  via  local  experts  reporting  back  to 
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their  colleagues,  the  survey  results  present  a  more  ambiguous  picture.  Due  to  the 
3 -year  time  period  of  the  grant,  penetration  of  non  curriculum  patient  safety 
material  could  not  be  guarded  against.  The  rigorous  evaluation  of  the  program 
itself  suffered  from  these  confounding  influences. 


Discussion 

Module  structure  and  content 

CME  has  undergone  significant  changes  over  the  last  20  years  or  so.  The 
focus  has  shifted  from  an  emphasis  on  transmitting  content  through  lectures  to 
facilitating  problem-based  learning.  ’  The  structure  of  the  modules  reflects  this 
development.  The  theoretical  overview  sets  the  stage  and  fits  the  traditional  CME 
paradigm  of  strict  knowledge  transfer.  This  overview  serves  as  the  foundation  for 
further  learning.  The  clinical  cases,  as  previously  mentioned,  illustrate  the  theory 
applied  to  reality.  This  translation  of  theory  to  clinical  practice  highlights  the 
usefulness  of  the  theoretical  paradigm.  Finally,  the  take-home  points  reinforce  the 
lessons  learned  during  the  interactive  case  analysis  with  concrete  suggestions  on 
how  to  implement  patient  safety  messages  in  the  physicians’  practice 
environment. 

As  anticipated,  physicians  identified  with  the  clinical  cases  in  each  module. 
Each  individual  case  exemplified  the  context  (read:  system)  in  which  the  problem 
occurred.  Prior  experience  with  similar  situations  helped  physicians  to  see  the 
material  as  relevant.  The  key  safety  messages  of  the  module  had  to  be  applied  to 
successfully  process  and  analyze  the  discrete  steps  that  led  to  the  patient  safety 
problem.  In  doing  so,  the  new  patient  safety  schemata  became  incorporated  into 
the  physicians’  repertoire  of  decisionmaking. 14 

The  take-home  points  of  all  seven  modules  were  compiled  into  a  single 
pamphlet  that  has  proven  quite  popular.  The  guide  was  not  offered  as  a  matter  of 
course  to  CME  attendees.  They  received  take-home  points  specific  to  the  module 
they  studied.  We  developed  the  pamphlet  to  distribute  at  our  patient  safety  booth 
at  the  2003  annual  session,  and  the  pamphlet  is  posted  on  our  patient  safety  Web 
site  (www.acponline.org/ptsafety/).  Despite  limited  exposure,  we  have  processed 
over  5,000  requests  for  the  pamphlet,  primarily  from  medical  directors  of  clinics 
and  hospital  patient  safety  officers.  All  seven  modules  can  be  viewed  on  the  Web 
site  in  PowerPoint  format,  and  the  site  offers  additional  resources  for  learning 
more  about  patient  safety.  The  curriculum  content  therefore  can  be  obtained 
without  CME. 


Summary 

Overall,  the  evaluation  of  the  CME  program,  Patient  Safety:  The  Other  Side  of 
the  Quality  Equation,  might  indicate  that  immediately  following  instruction, 
physicians  showed  positive  gains  in  attitudes  and  beliefs  that  patient  safety  issues 
are  relevant  to  clinical  practice.  However,  6  months  after  CME  instruction  the 
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apparent  effects  of  instruction,  on  physicians ’attitudes,  beliefs,  and  reported 
behaviors  were  less  consistent. 

There  are  several  limitations  that  may  have  contributed  to  the  ambiguous 
findings.  First,  randomization  either  to  instruction  or  to  control  conditions  was  not 
possible  in  this  evaluation.  Therefore,  self-selection  and  previous  history  biases 
are  major  concerns.15  Second,  though  efforts  were  made  to  control  for  information 
contamination,  there  was  no  practical  way  to  accomplish  this.  The  increase  in 
information  use  scores  by  all  participants  indicated  that  the  patient  safety 
“message”  was  spreading  via  media,  professional  journals,  etc.  The  physicians 
participating  in  the  CME  may  be  viewed  as  early  adopters  of  patient  safety  ideas; 
however,  there  was  insufficient  information  to  make  that  determination. 

Despite  these  apparent  limitations,  the  patient  safety  CME  was  successfully 
implemented  in  numerous  geographic  locations  in  the  United  States.  This  fact 
alone  demonstrates  the  wide  acceptance  among  physicians  that  patient  safety  and 
the  reduction  of  medical  errors  in  clinical  settings  are  important  issues  that  must 
be  addressed  in  professional  practice.  However,  recent  research  suggests  that 
many  physicians  still  need  to  get  the  patient  safety  message.  One  national  mail 
survey  of  physicians  (N  =  83 1)  conducted  between  April  and  July  2002  asked, 
“How  often  are  preventable  medical  errors  made?”16  Eighty  percent  indicated 
“not  very  often”  or  “not  often  at  all.”  More  recently,  a  telephone  interview  of 
ACP  members  (N  =  239)  conducted  between  June  and  September  2003  included  a 
question  about  the  threat  of  medical  errors  in  private  practice.17  When  asked, 
“How  would  you  characterize  the  threat  to  patients’  safety  in  private  practice 
settings?”  53  percent  indicated  either  a  somewhat  low  threat  or  a  very  low  threat. 
These  results  point  to  a  continuing  need  for  patient  safety  training. 


Conclusion 

The  main  objective  of  this  research  was  to  establish  an  ambulatory  care 
curriculum  for  the  average  physician  in  practice  and  to  disseminate  this 
information  widely.  This  goal  was  accomplished  by  relying  on  established  models 
of  academic  detailing. 1 1  The  recommended  behavior  changes  had  to  be  easy  to 
implement  without  requiring  substantial  resources  of  time  or  money.  The 
presenters’  survey  responses  attest  to  the  curriculum’s  success  in  this  area. 
Presenters’  testimony  refers  to  the  intuitive  nature  of  the  clinical  cases.  Indeed, 
this  was  the  one  feature  they  wanted  to  duplicate  the  most. 

To  meet  the  goal  of  reaching  out  to  all  chapters,  the  College  activated  its 
governing  infrastructure  and  acted  as  change  agent.  The  College  leadership  and 
their  chosen  presenters  acted  as  trusted  peers  and  influential  others  to  facilitate  the 
spread  of  the  program,  as  did  back-up  staff  at  the  College.  The  patient  safety 
messages  embodied  in  the  curriculum  found  a  ready  audience  that  relied  on  the 
long-established  integrity  of  the  College’s  educational  materials. 

While  dissemination  proceeded  smoothly,  the  very  success  of  dissemination 
undermined  the  statistical  rigor  of  the  research  design  for  evaluation.  Whenever  a 
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choice  had  to  be  made  between  dissemination  and  protecting  the  research  design 
from  contamination,  dissemination  won.  A  number  of  factors  combined  to  create 
a  minefield  of  confounding  variables.  The  entire  medical  community  was  exposed 
to  media  reports  and  professional  articles  on  patient  safety.  Thus,  there  could  be 
no  true  control  group.  Further,  the  voluntary  nature  of  CME  program  selection 
produced  a  biased  audience.  Moreover,  since  individual  chapters  selected 
different  modules,  content  evaluation  was  a  mixture  of  responses  to  different 
patient  safety  messages.  The  original  grant  application  naively  assumed  that 
chapters  would  devote  a  half-day  of  training  to  three  modules  at  a  time  and  that 
training  would  be  repeated  when  all  the  modules  where  completed.  This  did  not 
happen.  Instead  each  chapter  selected  one  module  for  training.  This  factor  alone, 
not  to  mention  a  given  presenter’s  charisma,  or  lack  thereof,  undoubtedly 
confounded  the  results. 

On  balance,  this  program  was  a  success  as  interest  in  the  program  continues 
beyond  the  original  target  audience.  The  translation  of  patient  safety  concepts  into 
clinical  cases  is  the  heart  of  this  program  and  found  universal  appeal  with 
speakers  and  audience  members  alike.  This  indicates  that  most  physicians,  as 
teachers  and  as  learners,  value  CME  that  is  directly  applicable  to  their  practice. 
Also,  modules  on  medication  errors  and  systems  focus  on  basic  information 
regarding  what  constitutes  an  error,  an  area  many  physicians  still  struggle  with. 
Future  studies  may  want  to  concentrate  more  fully  on  the  practical  usefulness  of 
patient  safety  concepts  and  move  away  from  attitude  and  behavior  questions. 
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